<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>GROOPS - Autoregressive model</title>

    <!-- JQuery and Popper -->
    <script src="https://code.jquery.com/jquery-3.4.1.slim.min.js" integrity="sha384-J6qa4849blE2+poT4WnyKhv5vZF5SrPo0iEjwBvKU7imGFAV0wwj1yYfoRSJoZ+n" crossorigin="anonymous"></script>
    <script src="https://cdn.jsdelivr.net/npm/popper.js@1.16.0/dist/umd/popper.min.js" integrity="sha384-Q6E9RHvbIyZFJoft+2mJbHaEWldlvI9IOYy5n3zV9zzTtmI3UksdQRVvoxMfooAo" crossorigin="anonymous"></script>

    <!-- Bootstrap -->
    <!-- https://getbootstrap.com/docs/4.1/examples/ -->
    <script src="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/js/bootstrap.min.js" integrity="sha384-wfSDF2E50Y2D1uUdj0O3uMBJnjuUD4Ih7YwaYd1iqfktj0Uod8GCExl3Og8ifwB6" crossorigin="anonymous"></script>
    <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/css/bootstrap.min.css" integrity="sha384-Vkoo8x4CGsO3+Hhxv8T/Q5PaXtkKtu6ug5TOeNV6gBiFeWPGFN9MuhOf23Q9Ifjh" crossorigin="anonymous">

    <!-- Mathjax -->
    <script type="text/x-mathjax-config">
    MathJax.Hub.Config({tex2jax: {inlineMath: [ ['$','$'] ],
                                  displayMath: [ ["\\[","\\]"] ],
                                  processEscapes: true},
                        TeX:     {Macros: {M: ["{\\mathbf #1}",1]},
                                  equationNumbers: {autoNumber: "all"}  }});
    </script>
    <script async src="https://cdn.jsdelivr.net/npm/mathjax@2.7.7/MathJax.js?config=TeX-AMS_CHTML" integrity="sha384-e/4/LvThKH1gwzXhdbY2AsjR3rm7LHWyhIG5C0jiRfn8AN2eTN5ILeztWw0H9jmN" crossorigin="anonymous"></script>

    <!-- lunr -->
    <script src="https://cdn.jsdelivr.net/npm/lunr@2.3.8/lunr.min.js" integrity="sha384-vRQ9bDyE0Wnu+lMfm57BlYLO0/XauFuKpVsZPs7KEDwYKktWi5+Kz3MP8++DFlRY" crossorigin="anonymous"></script>

    <!-- Mustache -->
    <script src="https://cdn.jsdelivr.net/npm/mustache@4.0.1/mustache.min.js" integrity="sha384-0PLEZVBpOQ+Kqw3anJWSNWvRxpEFt02tSpBvyRsA4WcvX/OTldWdXxGLVLvh954H" crossorigin="anonymous"></script>

    <!-- GROOPS Stylesheet -->
    <link rel="stylesheet" href="static/groops.css"/>

    <!-- icon -->
    <link rel="icon" href="static/groops_icon.png">

</head>
<body>
    <header>
        <nav class="navbar navbar-expand-lg navbar-dark bg-dark">
            <div class="container">
              <a class="navbar-brand" href="index.html"><img class="logo" src="static/groops_white.svg"></a>
              <div class="collapse navbar-collapse" id="mainNavbar">
                <ul class="navbar-nav mr-auto">
                  <li class="nav-item">
                    <a class="nav-link" href="programType.html">Programs</a>
                  </li>
                  <li class="nav-item">
                    <a class="nav-link" href="classes.html">Classes</a>
                  </li>
                  <li class="nav-item">
                    <a class="nav-link" href="general.parser.html">Parser</a>
                  </li>
                  <li class="nav-item">
                    <a class="nav-link" href="general.fileFormat.html">File Formats</a>
                  </li>
                </ul>
              </div>
                <form class="form-inline my-2 my-lg-0" action="search.html" id="searchTools">
                  <input class="form-control mr-sm-2" placeholder="Search" name="searchTerms" method="GET" value="" type="text" id="searchBox">
                  <button class="btn btn-secondary my-2 my-sm-0" type="submit" id="searchButton">Search</button>
                </form>
              <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#mainNavbar" aria-controls="mainNavbar" aria-expanded="false" aria-label="Toggle navigation" style="">
                <span class="navbar-toggler-icon"></span>
              </button>
            </div>
        </nav>
    </header>
    <main class="container px-0 pt-2 top-buffer">
        <div id="content" class="container">
            <h1 id="fundamentals.autoregressiveModel">Autoregressive Models</h1><p>
A multivariate (or vector) autoregressive model is one possible representation of a random process.
It specifies, that the output at epoch $t$ depends on the $p$ previous epochs, where $p$ is denoted process order,
plus a stochastic term.
In the following, finite order vector autoregressive - VAR($p$) in short - models as implemented in GROOPS will be described.</p><p><h2>Definition</h2><p></p><p>A finite order VAR($p$) model is defined as
\[
  \mathbf{y}_e(t_i) = \sum_{k=1}^p \mathbf{\Phi}^{(p)}_k\mathbf{y}_e(t_{i-k}) + \mathbf{w}(t_i),
  \hspace{5pt} \mathbf{w}(t_i) \sim \mathcal{N}(0, \mathbf{\Sigma}^{(p)}_\mathbf{w}),
\]where $\mathbf{y}_e(t_i)$ are realizations of a random vector process
Subtracting the right hand side and substituting the stochastic term $-\mathbf{w}(t_i)$ with the residual $\mathbf{v}(t_i)$ gives us
\[
  \mathbf{0}  = \mathbf{y}_e(t_i) - \sum_{k=1}^p \mathbf{\Phi}^{(p)}_k\mathbf{y}_e(t_{i-k}) + \mathbf{v}(t_i)
\]which can be used as pseudo-observation equations in the determination of the parameters $\mathbf{y}_e(t_i)$.
In matrix notation this reads
\[
  0 =
  \begin{bmatrix}
    \mathbf{I} & -\mathbf{\Phi}^{(p)}_1 & \cdots & -\mathbf{\Phi}^{(p)}_p \\
  \end{bmatrix}
  \begin{bmatrix}
    \mathbf{y}_e(t_i) \\
    \mathbf{y}_e(t_{i-1}) \\
    \vdots \\
    \mathbf{y}_e(t_{i-p}) \\
  \end{bmatrix}
  + \mathbf{v}(t_i).
\]After rearranging the vectors $\mathbf{x}_t$ to have ascending time stamps
\[
  0 =
  \begin{bmatrix}
    -\mathbf{\Phi}^{(p)}_p & \cdots & -\mathbf{\Phi}^{(p)}_1 & \mathbf{I} \\
  \end{bmatrix}
  \begin{bmatrix}
    \mathbf{y}_e(t_{i-p}) \\
    \vdots \\
    \mathbf{y}_e(t_{i-1}) \\
    \mathbf{y}_e(t_i) \\
  \end{bmatrix}
  + \mathbf{v}(t_i)
\]For practical purposes, the residuals above are further decorrelated using the
inverse square root of the white noise covariance matrix, leading to
\[
  \bar{\mathbf{v}}(t_i) = \underbrace{\mathbf{\Sigma}^{(p)^{-\frac{1}{2}}}_\mathbf{w}}_{=\mathbf{W}}\mathbf{v}(t_i), \hspace{25pt}  \bar{\mathbf{v}}(t_i) \sim \mathcal{N}(0, \mathbf{I}).
\]The used square root is in principle arbitrary, but should satisfy $\mathbf{W}^T\mathbf{W} = \mathbf{\Sigma}^{(p)}_\mathbf{w} $.
This means that both eigendecomposition based roots and Cholesky factors can be used.
After the applying the matrix from the left, we arrive at the observation equations
\[
  0 =
  \begin{bmatrix}
    -\mathbf{W}\mathbf{\Phi}^{(p)}_p & \cdots & -\mathbf{W}\mathbf{\Phi}^{(p)}_1 & \mathbf{W} \\
  \end{bmatrix}
  \begin{bmatrix}
    \mathbf{y}_e(t_{i-p}) \\
    \vdots \\
    \mathbf{y}_e(t_{i-1}) \\
    \mathbf{y}_e(t_i) \\
  \end{bmatrix}
  + \bar{\mathbf{v}}(t_i)
\]which yields fully decorrelated residuals.
Currenty, VAR($p$) models are saved to a single  file which contains this matrix.
</p>

        </div>
    </main>
</body>
</html>
